Published as:

Science and technology policy in the developing world involves special problems since much of the financial support for S&T originates outside the countries where research is done. The development of information for policy and strategic planning decisions is therefore critical for national research policy makers, international organizations, and donors. However, prior attempts have been plagued by serious methodological problems. We describe a multi-faceted approach for generating systematic information on scientific and technological institutions in developing countries based on the concept of the research system as a multi-organizational network. By providing an account of how this approach was implemented in a three country study we hope to shed light on several related problems in developing information for policy. First, how can relevant actors in research systems be identified? Second, how can a national research system be systematically surveyed? Third, how can system-wide estimates of organizational linkages and performance be obtained? INTRODUCTION Science and technology policy in the developing world is a paradox. Since the science and technology systems are institutionally simpler than in developed countries, with relatively fewer objectives, it might seem that policy and strategic planning would be a straightforward affair. Of course, that is far from the case. In fact, the process of setting science and technology policy is more complex. Since (1) much of the financial support for S&T originates outside the countries where research is done and (2) donors, multinational corporations, and international organizations continue to maintain a diversity of goals and interests in developmental issues, it would be truer to say that S&T policy does not have its institutional locus "within" the country, at least not in the same way that developed countries do. Thus, with a variety of stakes in the pace, direction, and content of research, national research policy-makers, international organizations, and donors all require systematic information for policy and strategic planning decisions. It is imperative that such information not rely exclusively on the official documents, formal organization charts, and annual reports that are outdated, inaccurate, and unavailable. A variety of substantive issues are important in the contemporary policy context. For the improvement of research capacity the perceived needs of actors in the developing world are crucial. This "demand side" of the capacity equation must be addressed if a policy of building research capabilities is to be constructed with the assistance and input of developing countries themselves. The rise of "demand-based" considerations have been identified as a shift in policy development (Biervliet 1994, p. 23) and has been adopted as a key component of the new Dutch research policy. Relations between researchers and between organizations are generally considered important for the efficiency and effectiveness of the research enterprise. The kinds of linkages 2 generally deemed to be important are those between researchers themselves but recent reports stress that stronger linkages must be developed between national research systems and policymakers, extension services, development agencies, and the international community as well as farmers (Taylor 1990). Moreover, the emergence of the nongovernmental organization as an increasingly important institution for adaptive research makes it imperative--albeit difficult--to consider their role in a national research system (Farrington and Bebbington 1993). In this context that the Dutch Advisory Council for ScientificResearch in Development Problems (RAWOO) requested a study of the research systems in three locations chosen to represent countries at different levels of development: Ghana, Kenya, and the State of Kerala located in southwest India. The general objective of the study was to describe and assess the enabling environment within the fields of agriculture, natural resource management, and environmental research. By providing basic empirical information on researchers, their organizational context, their research networks, and their perceived needs for improving the research system we sought to assist in the development of RAWOO policy recommendations for building research capacity The selection of these fields owed primarily to the fact that they represent major areas of interests for each of the major policy groups (national and international). Moreover, most of the research undertaken in these countries is related to these fields. A major emphasis was the development of methodology. In this essay, we describe a multi-faceted approach for providing systematic information on science and technology in developing countries. It is based on a empirical survey that is designed for the organizational systems within particular countries while recognizing the common international context within which all operate. The orienting framework for the study is 3 the idea of the research system as a social network (Schott 1993; Shrum and Bankston 1993/94). These technical systems may be defined as a set of organizations (and relations between organizations) organized around the solution to a particular problem or set of related problems (Shrum 1985). Several problems have impeded the development of sound information for policy. First, how can relevant actors in the research systems be identified? Second, how can a national research system be systematically surveyed? Third, how can estimates of organizational linkages and performance be produced that accurately tap the all-important informal dimension of networking and are not irredemably biased by the self perceptions of researchers? We provide a rationale and suggestions for addressing these questions by means of an account of how these issues were resolved in the three country study. In the first section, we show how actors were identified and productivity measures developed from a comprehensive bibliographic search. These results were found to be only partially adequate. In the next section a process of sampling by sector and organization is described that provides a systematic representation of linkages within the research system both internal and external to a country. Finally, we show how the manipulation of questionnaire data on organizational ties allows the assessment of the different types of tie and relations between institutional sectors, as well as the measurement of performace. Such measures provide a focused representation of institutional position and capacity. BIBLIOGRAPHIC STUDY OF RESEARCH SYSTEMS Most approaches to the identification of institutions and researchers within developing 4 country research systems rely on a limited number of internationally available data sources such as the Science Citation Index produced by the Institute for Scientific Information. From such data bases measures of productivity and performance are developed from counts of publications and citations. Such sources have been widely criticized for their underpresentation of developing country publications. Further, much LDC research is not focused on "basic" scientific areas that are typically included and thus would not be expected to be found there. The problem may be stated as follows: how, and to what extent are bibliographic sources useful in studying LDC research from outside the LDCs? The general answer is that current bibliographic search capabilities make it possible to generate useful measures and relational databases that provide a partial overview of organizational research in LDCs, but do not usefully serve to identify the researchers and measure performance. A bibliographic approach to LDC research requires identification of data bases and search fields. At a superficial but useful first approximation, computerized searches using such database providers such as DIALOG can be used to identify data bases with coverage in particular areas and determine relative numbers of publications at very little cost. Though there are differences in coverage, each data base provides titles, authors, and complete references to articles, books, reports, and other published and unpublished items. Most provide abstracts and subject keywords in addition. Each data base covers hundreds, sometimes thousands, of journals. Initially, such a facility can be used to get an overview of the kinds of research undertaken in a target country and to identify institutional actors for sampling. This benefit alone should make such online searches a first step in any policy study, particularly since measures of international interest can be constructed together with the identification of 5 databases. An inexpensive search can be accomplished independently of specific identification of bibliographic records themselves. For example, a preliminary search may be conducted using the ALLSCIENCE facility in DIALOG, which provides an overview of over 160 databases concerned with all fields of science and engineering. By searching the Corporate Source field, containing the organizational affiliation and address of authors of entries in these data bases, the frequency of entries (publications, reports, conference papers) originating in a particular country can be tabulated for each data base over a particular period of years. In the present case, since the subject matter of interest encompasses a large proportion of the total research profile of many developing countries, and since specialized reference bibilographies exist in these areas, it seems apparent that if any such fields can be accurately assessed with bibilographic sources, it would be agriculture and natural resource management. After conducting a number of test searches without downloading any references and identifying the most relevant data bases, seventeen international databases were searched for the 1992-1993 period. It is useful for some purposes to divide the data bases into a general type that seeks relatively comprehensive coverage (e.g., SCISEARCH ) and field-specific data bases (such as CAB Abstracts, AGRIS International, and AGRICOLA for agricultural topics). When general databases are selected, classification codes are available in almost all cases to subset the records and allow some selection by topic. The Corporate Source fields were used to indicate the organizational and national affiliation of the author. The Basic Index (title, abstract, keywords) was used to indicate the subject of the item. Even this extensive search does not come close to identifying all 6 publications produced by researchers within a country, as we discovered after beginning field work. What it does reveal in relatively comprehensive fashion are (1) the kinds of work accessible to scientists and readers elsewhere (2) the contrast between the amount of research work generated within the country and the work done elsewhere. Two files for each location (Ghana, Kenya, and Kerala) were developed, one for publications within the country, that is, authored by individuals who were affiliated with organizations in the target location. A second file was developed for publications about the location, that is, which used aspects of the place as subject matter or collected data there for scientific purposes. The first file was simply created by using the corporate source field in each database, which indicates the organizational affiliation and location of the author(s). The second file was developed by searching the basic index for the location of interest, then eliminating all those items that were also identified in the corporate source search. This remainder gives items on or about the country that do not originate within the country. These comparisons allow construction of estimates of international interest and dependency that can be useful for policy purposes. First, how much North-based research on a country has been conducted for a particular period of time? Second, how much external interest is there in a location relative to its own research capacity? The basic subject index (minus the items included through the corporate source fields) is a measure of the extent to which there is research in internationally available databases that pertains to a location but originates outside the location. Hence, it may be used as a measure of international interest for a given country or location. The usefulness of this measure may best be seen in relation to the research generated from 7 within the location (i.e., South-based research). The ratio of internal to external publications shows the degree to which scientists within a location generate their own research. Hence, it may be used as a measure of external dependency, with lower ratios indicating higher levels of dependency. High ratios mean that researchers within a location contribute more of the output relative to external scientists. The core idea is that when the count of publications by scientists within a country is compared with external publications (publications about the country that are not published by its own scientists) it is possible to rank countries in terms of underor overcommitment by the donor community and developed country scientists. In general the results are consistent with the history of external donor interest in Kenya and the relatively low profile of Kerala--we expected to find that Kenya has relatively high visibility, while Kerala has relatively low visibility in the international scientific community. Indeed, Kenya exhibits the highest level of productivity as measured by these internationally available sources (articles, papers, books, reports), both domestically and internationally. This means it produces the most and gleans the most international interest (above both Ghana and Kerala). Kerala, which is a relatively small Indian state closer in size to Ghana than India, produces twice as much as Ghana, but only about half as much as Kenya. Still, there is substantially more international interest in Ghana than Kerala. Taking the ratio of internal to external publications shows that relative to the others, research in Kerala is much less dependent on the international community. Indeed, researchers in Kerala produce two to three times as much as external researchers about the state. (Ghana=.86; Kenya=1.1; Kerala=2.7). This is consistent with other data that shows although the Keralan research system is relatively advanced, its products are relatively unknown outside of India and its scientists are relatively isolated. 8 The next step is to download the records identified through the search (eliminating duplicate records) into a DOS (ascii) file. Though a database provider seeks consistency in formatting, each data base is separately designed and marketed. Hence, although there is a good deal of similarity in the fields used across files (author, title, keyword), there remains significant variation. What can be done? One solution is to maximize the use of the individual fields in each bibliographic record to create a single relational data bases with the output. This requires a separate computer program but is well worth the effort to consolidate all the records in a common format that can be sorted by any field and provides a rapid overview of the published research of most organizations. After completing fieldwork in the three countries it seems clear that the bibliographic search is at best adequate for the identification of organizations but not individuals. If no information is available other than the bibliographic search--even if the search is only conducted for such a limited period (in this case, two years) the most significant institutions within the country can generally be identified. However, using this methodology (assuming it to be the only means available) to identify the most prominent or productive individuals is problematic. In sum, a bibliographic search and the creation of a relational database are useful in three ways. First, they allow preliminary identification, before going into the field, of many of the principal organizations in agriculture and environmental research in each location. Next, they provide an estimate of external interest in particular countries as well as the size of the internationally visible research community (writing on and about the country by other, mainly developed, countries). Finally, it can be extremely important in the field, allowing the identificatio of some respondents for the survey--sometimes in the presence of the institute 9 directors themselves! However, our experience confirms the sceptical view of bibliographic methods that argues reliance on internationally available sources will give at best an incomplete picture and at worst a misleading view of the research system in LDCs. FIELD STUDY OF RESEARCH SYSTEMS A full description of the orgnanizational and policy context for natural resource management and sustainable agriculture cannot be restricted to the internal characteristics of the organizations themselves. The social network approach is premised on the idea that contextual factors are crucial for the explanation of the behavior of social entities such as organizations: it is not simply the characteristics of the entities themselves that are important, but rather their relationships with other entities that constitutes context. As applied to contemporary S&T policy the network approach is especially relevant, for relationships (a) within national research systems and (b) between North and South are a key concern. In the early stages of the project, it was hoped that the bibliographic phase just described could also provide an overview of NorthSouth research linkages. However, even such a comprehensive search proved too restrictive in coverage of the research in the three states representing varying levels of development. The field survey of researchers was conducted to determine the institutional conditions for research in developing countries. The survey instrument included both structured and unstructured sections in order to capture the major dimensions of research activities, international and national organizational contacts, frequency of discussions with various groups, supervisory roles and local contacts, professional memberships and activities, selfreported productivity, attitudes on agricultural and environmental issues, and the needs of the research system. 10 The most extensive part of the interview pertained to the respondent's networks, characterizing both internal linkages within each research system and external relations--in particular, linkages with other countries and international organizations. Network instruments are difficult to design for many policy-relevant domains for the following reason: unless a complete list of the people or organizations that constitute a system already exists, the true set of possible relations will not be known until after the study is completed (hence the popularity of open-ended items asking the respondent to list his or her contacts). In the present case, it was possible to assemble a list of international organizations, assuming that the international context would be the same for all three locations. Lists of national research institutes and universities are, of course, specific to each country. Moreover, because of the importance of NGOs as new institutional actors in the policy context of sustainable agriculture and natural resource management, there is no alternative to intensive background work on arrival in each location. Several days were spent in each of the capitol cities interviewing informants and determining which NGOs would be appropriate both for inclusion in the sample and the contact list. Since this list of organizations should constitute the core institutional matrix in which each individual research unit is embedded, its specification is the single most important task in the policy study apart from the selection of the organizational sample itself. Development of Organizational Contact Lists and Sample Two lists of organizations are required. First, a relatively comprehensive group of organizations is needed to represent each state research system. This is used to create a 11 structured format for detailed questions about linkages. Of course, a fully comprehensive list would be impossibly long, but we sought to be relatively inclusive in generating a list of the most important organizations in agriculture, natural resources management, and environment work. In practice, most respondents have a surprisingly high tolerance for reviewing such lists. Second, a subset of that list was designed to form the basis for the sample of organizations to be interviewed. This consisted of a more selective list of organizations chosen based in part on the bibliographic search described above. Both the contact lists and the final sample are stratified by sector: international organizations, government ministries, universities, national research laboratories, private firms, and NGOs. The general strategy was to prepare a preliminary list before arrival using the relational database. In Ghana and Kenya we had the advantage of prior country studies by ISNAR (International Service for National Agricultural Research). This information was supplemented with unpublished materials available locally as well as nominations from informants. For the national institutes and university departments, we began by creating a sectoral code, sorting the database by sector, and selecting organizations that appeared more than once. The configuration of "official" research institutions in the area of sustainable agriculture and natural resource management is somewhat different in each country. International Organizations{tc \l3 "International Organizations} For the contact list the complete list of CGIAR centers was used, supplemented by international research centers that are not in the CGIAR system (e.g. ICIPE). We then added the major international donors and foundations, including multilateral agencies and bilateral programs, some major "networks" (e.g., SPAAR). For the sample itself, such international 12 organizations were restricted to Nairobi, Kenya, which has probably the largest concentration of international research centers in the developing world. These include the International Center for Research on Agroforestry, the International Laboratory for Research on Animal Diseases, the International Center for Insect Physiology and Plant Ecology, and the United Nations Environmental Program. Since Kerala is not a country but a state, the research organizations under the central government (ICAR, CSIR) play a structurally similar role. Universities{tc \l3 "Universities} Both the contact list and the sample were relatively comprehensive in each location owing to the small number of universities involved. For sampling purposes, as well as for the contact list itself we used university departments rather than universities as organizational units. For example, in Kenya, the five universities (Nairobi, Moi, Egerton, Jomo Kenyatta, and Kenyatta) were used on the contact list. Departments were selected using a criterion of more than two appearances in the database and spaces were provided for other departments in case the respondent had contact with departments other than those listed. For the sample itself, we selected four out the five universities, excluding only Kenyatta owing to our focus on environment, agriculture, and natural resources. In Kerala, all four major universities were included (Kerala Agricultural University, University of Kerala, Cochin University, and Calicut University). One college that appeared in the bibliographic search was added to the contact list. In Ghana the three major universities were listed and sampled (Cape Coast, University of Ghana Legon, and Kumasi). National Research Organizations {tc \l3 "National Research Organizations } For each contact list we began with all government institutes that were significantly 13 involved in agricultural or environmental work. In most countries, this involves identification of the appropriate ministries, research councils, or other administrative bodies. Several other categories of organization can fruitfully added to the contact list and could profitably be sampled in a more comprehensive study: large private enterprises with a research component, important consulting firms, institutes under commodity boards, relevant ministries, botanical gardens, and other resource, information, and testing centers. For the sample itself the relational database constructed through bibliographic means can be helpful in the case of too many national institutes to interview in each one. We used a general criteria of two authors/items to narrow the field before developing area clusters for the program. Non-Governmental Organizations{tc \l3 "Non-Governmental Organizations} For NGOs, unlike universities and national institutes, the difficulty is threefold: (a) identifying NGOs ; (b) determining their substantive (project) focus; and (c) their actual or potential role in research. A relatively large contact list can be developed from a variety of sources, prior to focusing on NGOs that actually conducted some form of research--though the activities that are defined as "research" fall outside the scope of research as defined by national research institutes. These involvements may be minimal and some are incipient, but our results confirm the views of Farrington, Bebbington, and colleagues (1993) on the recent emergence of NGO research. The final selection of NGOs for inclusion in the sample was based not only on their substantive focus (agriculture, environment) but also based on the extent to which they were involved in research. A variety of methods, including organizational nominations and public and private documents, was used both before and after arrival for the field study, but the resources generally available, together with their advantages and disadvantages can be 14 summarized. (1) Bibliographic searches of internationally available databases as described above. The bibliographic method is particularly unsuited for the identification of NGOs. Occasionally, institutional affiliations from this sector are present, but it is not an exaggeration to say that no significant research efforts of NGOs in the three states were uncovered through our comprehensive bibliographic search. (2) Lists of local NGOs provided by development agencies or northern NGOs, subscribers to newsletters, registries in international directories, and institutional affiliations of conference attendees at relevant international conferences. The main advantage of these sources is their availability but they are often outdated, unsourced, and misleading. Indeed, these sources are particularly prone to the problem of "phantom NGOs" consisting, at their worst, of one or a few individuals without equipment, projects, or knowledge. (3) NGO lists maintained by state offices (national registries, NGO coordination bureaus, councils, etc.). These sources are essential but useful to varying degrees. Some are relatively comprehensive, while others (such as one compiled by a state environment protection council) are extremely inadequate. While there is sometimes an effort made to maintain a current list there are often striking omissions. (4) Nominations by offices of state agencies, national associations of NGOs, or NGOs with a significant national presence, personnel in extension services, consulting firms--in short, people with experience in the field who could plausibly claim to provide an overview. This preferred method requires identifying individuals in appropriate structural positions and using them as key informants to identify the most important voluntary organizations. The main 15 drawback is the time required to identify these positions and meet with the individuals. It may seem odd to mention state agencies and NGOs in the same breath here, given their sometimes conflictual relations, but all informants have biases. NGOs themselves are often in structural competition and NGO informants, like certain representatives of the state, may omit relatively important organizations when providing an overview of the national scene. A combination of all of these resources can be used to create a list of NGOs. As the list is generated, each informant is asked to review it for those organizations that are "most interested in" or "have some involvement in" research as well as offer other nominations. Those organizations to appear on the final contact list should be those indicated in multiple sources as non-governmental organizations with a particular substantive interest (in our case, natural resources and agriculture) and a research interest. Hence an iterative process must be used, successively generating a catalog of NGOs, seeking to confirm research and substantive interests, and reviewing the catalog of organizations again with each new set of additions. The importance of reviewing and re-reviewing for thoroughness, mistakes, and duplications, cannot be emphasized too strongly. This procedure allows the incorporation of NGOs in the sample according to (1) the number of mentions (nominations) that were received by this variety of sources, (2) their degree of emphasis on or involvement in research and (3) a focus on agriculture, natural resources, and/or environmental issues. Interviews The preparatory work required approximately one week. A team of three interviewers (always including nationals) spent an additional month in each location, conducting a total of 16 293 structured interviews. The number of interviews per organization ranged from one to five and was proportional to the size of each organization: fewer interviews were conducted at NGOs and more interviews at large national institutes. A standard response rate is difficult to calculate owing to the method used to obtain the interviews. In each location we tried to conduct interviews at every significant organization in the state and NGO sectors. For university departments where a bibliographic criterion was used (see above). Twice we failed to get interviews at an organization, such that the response rate at the organizational level was close to 100%. Other missing data were not because of refusal but due to practical impossibility. When the organization is considered as the sampled unit, then we failed in only one case (a Kenyan NGO that had just incurred the loss of all their computer equipment owing to theft). In Ghana, one university professor terminated the interview without explanation after twenty minutes. Specific interviewees were identified after discussions with the director, assistant director, or head of department at each institution that explained the purposes of the study and discussed the bibliographic data base. We sought to interview at least one researcher who appeared on the bibliographic list and one researcher who was not on the list (mid-career individuals were preferred). Women researchers were included wherever possible and we tried to obtain a diversity of disciplines within our broad mandate. In general, outstanding levels of cooperation were attributable to the cooperation of the respondent's supervisors and the educational levels of the respondents. A "completed" interview was defined to include, at minimum, network and background questions. The most important determinant of the length of the interview was the number of contacts reported by the respondent. Large numbers of reported contacts require lengthy interviews because each reported contact necessitates two additional 17 questions pertaining to the types of tie involved in the relation and an assessment of performance. NETWORK MEASURES OF LINKAGE & PERFORMANCE In pursuing the conceptualization of the research system as an organizational network, the previous section described a survey methodology for the collection of information about linkages and outcome measures. Of course, generation of data by itself is insufficient for policy analysis. In the final section we present a practical account of some of the data manipulations that can be accomplished to construct indicators of key dimensions of the organizational network. The nodes and the units to which they are connected may be either individuals or organizations. In the following analysis the linkages--that is, the contacts reported by individual respondents--are always with organizations. But this information can be used to indicate two levels of linkages. We first illustrate the individual level (the simplest and most direct method) by using the individual researchers who report the ties. Next we consider ways in which their reports may be aggregated to the level of the organization. Of course, we adopt the network notion that the linkages that characterize researchers and institutions are more important determinants of the performance of these institutions than characteristics internal to the institutions themselves. Throughout, the focus is restricted to "ego" or "actor-centered" measures rather than "systemic" measures of the sort produced by network packages such as UCINET. Thus, the kinds of analysis that can be accomplished employ a data matrix (units of analysis by variables) similar to conventional survey techniques. Yet the measures of important dimensions--and of 18 course the conceptualization of the dimensions that are important--differ in important ways. By constructing our indicators relationally we can examine important policy ideas pertaining to the linkages between institutions. The units to which the focal node is connected are called alters. The measures to be illustrated here in detail relate to two principal aspects of the networks of actors in our sample. The first is composition, which indicates the average characteristics of the alters to which our focal nodes (individuals or organizations) are connected. The second is range, which indicates aspects of diversity in the networks to which focal nodes are connected. Both the average characteristics and the diversity of actors are important to the evaluation of a research system, but require further specification depending on the purposes at hand. Composition measures are important because the average kinds of linkages maintained are an indicator of the types of opportunities and resources to which an actor is likely to have access. Thus composition measures can inform us about the percent of ties involving information exchange or the percent of ties to universities. Range measures give us information about the concentration or dispersion of these relational types or sectors without reference to any particular relational type or sector. Range measures are important because networks with greater range tend to reach more deeply and extensively into the social structure. In contemporary global science such networks may be effective in tapping informational and other kinds of resources necessary to produce quality science and should also impact the relevance of the science that is pursued (Schott 1993). In simplest terms, an indicator of the range of a network is its size. Contacts with larger numbers of actors should offer access to a greater variety of resources and opportunities. The size measure is a simple count of the number of organizations to which a focal individual or 19 organization reports ties. Howevcr, sheer size is not the only indicator of the kinds and quality of relations that may be important. In what follows we suggest more refined measures. Consider first the composition and range of the organizations to which individuals are connected. This set of individual-level measures examines the sectoral location of the organizations with which a given researcher has contact. The sampled organizations need not correspond precisely (though they should represent a subset) to the contact network of interest, which may be much larger. There are six sectors in our contact network, representing the most significant institutional (organized) sectors in a research system : Universities, National Research Institutes, NGOs, Private Enterprises, International Organizations, and Ministries. For an individual respondent, composition measures are calculated as the proportion of his or her total ties with individuals in other organizations that lie in each of the six sectors. Thus if an individual has twelve organizational ties and four are reported with NGOs, the NGO composition of this individual’s network would be .33. Such measures tap the sectoral locations of an individual’s contacts with other organizations. Sectoral location becomes especially important once the respondent's own sector is taken into account. Then we can build a measure of network range in terms of whether the contact is within the same or a different sectoral locations. Range measures for the individual level of analysis can be constructed for heterophily, diversity among alters, and sectoral range. Heterophily measures represent the average difference between the characteristics of the focal unit and the characteristics of their alters. To construct a measure of sectoral heterophily, a measure must first be constructed that indicates whether each alter to whom an individual is connected is in a different sector (1) or the same 20 sector (0) as the focal individual. This indicator is then determined for the individual's personal network level by calculating the proportion of the focal individual’s ties that connect them with a different sector. If an individual in the NGO sector has 12 total ties and 6 of these ties are to organizations outside the NGO sector, the sectoral heterophily score would be .50. A second measure of range indicates the breadth of organizational sectors to which the focal individual has ties. We construct this measure by counting the number of sectors to which the focal individual is tied. Since there are six sectors on the contact lists, this measure will range from one to six, where a score of six indicates that the focal individual has a tie to each of the six sectors, while a score of one indicates ties to organizations in only one of the six sectors. An individual with twelve ties would need to have at least one tie to an organization in each of the six contact sectors (University, National, NGO, Private, International, and Ministries) to score the maximum of six. A second individual, also with twelve ties, might have all ties to universities, in which case they would score only one on the sector range measure. Our final measure of range indicates the dispersion of ties across sectors, or sectoral diversity. This is measured by the Index of Qualitative Variation (IQV), a measure of dispersion that shows the extent to which the ties of an individual are concentrated in one sector or evenly spread across the six sectors. The individual contact information is aggregated to the network level to evaluate the distribution of contact organizations. The IQV ranges from 0 to 100 to indicate the percent of the maximum possible spread of organizations across sectors that actually exists for a respondent. If all organizational contacts are in one sector (maximum concentration), the score on this measure is 0. The more evenly organizational contacts are spread across sectors (greater diversity), the higher the score to a maximum of 100. If an individual reports twelve 21 organizational contacts among six sectors, this measure would have a score of 100 if there were two contacts in each of the six sectors. A score of zero would obtain if all twelve contacts were in a single sector. The IQV differs from simple sectoral range, which is maximal when there is at least one organizational contact in each sector, regardless of the distribution of other contacts. For example, with twelve organizational contacts and six sectors, sectoral range is maximal whether the distribution is (7,1,1,1,1,1) or (2,2,2,2,2,2) because there is a contact with each sector in both cases. However, sector dispersion is maximal only in the latter case where there is an even spread of contacts across sectors. These measures are appropriate for testing different ideas. If, for instance, organizational performance is maximized with increases in sectoral range, the specific distribution of ties may not be important. However, it may be that the performance of the system as a whole is associated with a particular distribution. The second set of measures examines the composition and range of the types of ties to organizations. These ties, or relational types, indicate what is transmitted (transacted, exchanged) through the focal individual’s ties to other organizations. We asked about eight primary relational types. The eight tie types (see Appendix) are: receive funds, collaborate, friendship, worked at organization, visited organization, exchange of information, exchange of materials, and attend conference or workshop. (The residual category “other” is not used here.) Each linkage (relation) reported by an individual to an organization may consist of as few as one or as many as eight tie types. Composition measures are constructed by calculating the proportion of an individual’s ties to organizations in which a particular relational type is present. For example, if collaboration is 22 present in six out of twelve total organizational relations, the collaborative composition score would be .50. If information exchange is present in ten of twelve linkages then the information composition score would be .833. There is clearly overlap between collaboration and information since in at least four of the ties (and perhaps as many as six), both collaboration and information characterize the tie. That is, at least two relational types (information and collaboration) characterize four of the ties (in this example there are only two ties that lack information transfer). This aspect of network ties is described as multiplexity, the number of distinct exchanges or relational types that constitute a particular linkage. An individual tie might have all eight relational types. This particular configuration is atypical but the extent of overlap is an empirical question of interest since one idea that is commonly put forward in policy discussions is that limited linkages are sometimes created in workshops where people spend a great deal of time talking and little else. But it is also unusual that a linkage is characterized by only one relational type. In the above example, information is often associated with collaboration, so this overlap is a reasonable scenario. Multiplexity measures may be general or specific. General multiplexity indicates (a) the average number of relational types per tie, or (b) the proportion of a number of relational types per tie (that is, the proportion of all ties characterized by only one type, two-types, etc.). Specific measures of multiplexity indicate the proportion of all ties which contain certain relational types deemed to be important. From a policy perspective it may be important to know what proportion of all ties have components that combine, say, workshops with collaboration. (It may be specified that the ties contain just these relational types and no others, or that these 23 relational types are present regardless of what other relational types are also present in the tie.) Parallel to those measures that deal with sectoral location of ties, measures which indicate the range of relational types can be constructed. A general measure indicating the range of relational types would be a simple count of the number of tie types that are present. To construct this measure, one aggregates across all linkages the number of each relational type present in an individual’s total network. Since any single tie type may be present in each organizational tie; this measure ranges from one to eight. For example, an individual is tied to twelve organizations that exchanges information with ten would have a total of ten information relations in his network. Assume their distribution of tie types were the following: funds-2, collaboration-6, friends-2, work there-1, visit-2, information-10, materials-0, workshop-1. Then there are a total of 24 total relational types (2+6+2+1+2+10+0+1). Therefore the breadth of relational types is seven (seven of the eight tie types are present in the network). Parallel to the sectoral IQV measure is a useful measure of relational diversity (IQV) ranging from 0 to 100. It is maximal when the relational contents are spread evenly across the set of relational types and minimal when relational contents are all of one relational type. In a network with twelve organizational ties, 24 total relational contents, and eight distinct tie types (as in this instrument), relational diversity would be maximal if there were three relational contents in each of the eight relational types (the relational diversity score would be 1.0). With 24 relational contents and 12 organizational ties, all relational contents cannot be concentrated in one relational type, so the diversity measure cannot be zero. The most concentrated distribution of relational types would be (12,12,0,0,0,0,0,0) yielding a diversity score would be .57. For the distribution in the example above (2,6,2,1,2,10,0,1) the score would be .845. Thus this measure 24 is minimal when there is one relational content per tie and these relational contents are all of the same type. With 12 organizational ties and 12 relational contents, a distribution like 12,0,0,0,0,0,0,0 would result in a relational diversity score of 0. Representing Organizations from Surveys with Individuals This sample and instrument design fosters both an analysis of the networks of individuals and the networks of organizations themselves. Data records for each reported linkage are most efficiently and effectively entered in a unique data file separate from the records (observations) of individual respondents. The two files are easily and accessibly linked through identification codes assigned to each respondent in both the respondent data file and the linkage data file. However, the crucial identification codes are both the respondent's organization and the organization with which the linkage is reported, which are coded in each linkage record. The significance of these codes is that the organizational measures can then be constructed both from the standpoint of the reporting organization and the standpoint of the contact organization. Measures from the standpoint of the reporting organization (derived from individual reports) may be combined with other information aggregated from the respondent data file and other sources to create a richly textured body of information about a particular organization. But in one respect measures from this standpoint are less consistent with the central intuition of network analysis than measures developed from the standpoint of the contact organization. This central intuition involves (1) a suspicion of self-reported relations that is both realistic and prudent, together with (2) a faith in indicators derived from multiple reports by other organizations that reduce (alleviate, counterbalance) the multiplicity of biases that 25 characterize individual respondents. Few if any multiorganizational surveys can aspire to interview every individual in every organization. Just as a small number of individuals generally must represent a larger population of interest individuals must represent the organizations in which they are members. Essentially, by aggregating the respondents within each organization to an organizational level, we have information about the structure of interorganizational ties. The outstanding issue is the technique for aggregating organizational level measures from individual level responses. This issue, of course, is not specific to the science and technology system issues discussed here. All organizational survey responses must ultimately derive from individual persons. How shall these responses be treated such that they represent the organizations to which these individuals belong? We consider four solutions to the representation problem. Consider the sample from the State of Kerala in southeastern India. We interviewed fifteen organizations with a single respondent, eighteen with two respondents, twelve with three respondents, and three with four respondents. As discussed in the previous section, we sought to sample individuals in organizations roughly proportional to organizational size. The first and simplest solution is to sum the information from individuals within each organization to represent the characteristics of the organization as a whole. In some cases this might be a desirable result. But it can be argued that even though the number of respondents is related to the size of the organization, it is not an adequate proxy for organizational size and thus should not be used as a weighting factor. A second approach is to characterize the structure of organizational ties as being represented by the average network structure of the individual respondents in an organization. In 26 this approach it is only necessary to take the average of the network measures (discussed above) already calculated for members of each organization. This method can provide us with useful information in the sense that we characterize organizations by the average ‘collegial circle’ of organizational respondents. The third and fourth approaches are similar apart from the source of information utilized. The third approach establishes respondent organization-by-contact organization records with information about the presence or absence of each of the eight relation types. With this method redundant information is excluded. For example, if two informants in organization ‘A’ mentioned a ‘collaborative’ relation with organization ‘Z’, the respondent by contact organization record would simply contain information the presence of a tie between organization ‘A’ and organization ‘Z’--it does not matter that it was mentioned by two respondents. This method should minimize the effect of redundant reporting of a tie. At the same time, it should give us a better picture of the larger organizations which are the primary source of the multiple informants. We think it is reasonable to assume that in a small organization (particularly NGOs with a staff of only one or two persons) an individual can obtain sufficient information from a single informant. However, due to the complexity of larger organizations, one informant may not provide sufficient information about the overall structure of external organizational ties. So in this method the redundant information is excluded as network measures are built with respondent organization-by-contact organization records. The network size for a focal (respondent) organization would be the number of contact organizations mentioned by informants (respondents) in the focal organization. A fourth approach to developing measures of organizational network structure from this 27 database corresponds to the "central intuition" of network analysis. It requires that we reorient from the conventional source of information (the respondent organization) to information supplied about each organization by the individuals in other organizations who have reported contact with it. Each respondent organization in the sample is also a possible contact organization for other organizations. Thus, while we have information that respondent organization A gives us about its relations with other organizations, we also have the information from all organizations (B, D, F) that report relations with organization A. Since this sample contains respondents from university, NGO, and national organizations, we only have information from these types of organizations. Thus, we know what other university, NGO, or national institutes report about their relations with organization A. But we do not have information from private enterprises, ministries, or international organizations about their relations with organization A. Even given this limitation, the value of this source of information about organizational network structures lies in the fact that information is not based on ‘self-report’ by focal organizations but rather reports of relations from their alters (organizations that report contact with the focal organizations). To construct these measures we use the respondent organization-by-contact organization (discussed in the third method) but use the contact organization as our base organization. We then aggregate the information, for each contact organization, from all the respondent organizations which report ties to it and build our network measures. We are also able to construct an estimate of organizational performance based on the assessment of the focal organizations by their alters (contact organizations). The measures of organizational network structure we developed are the same as the set of measures we discussed for individual respondents. This is clear for the ‘individual average’ 28 method (number 2). For the respondent organization method (number 3) and the contact organization method (number 4), we merely replace the individual respondent as our focal point with either the respondent organization (method 3) or the contact organization (method 4) for the calculation of our network measures. We then construct measures of network composition, diversity, heterophily, size, and range. As with individual respondents, we can use information about either sector or relational type to construct these measures. When the respondent organization is the focal point, the sector-based measures refer to the sectoral location or distribution of contact organizations (six sectors). Network size in this case is the number of contact organizations with which the respondent organization reports ties. Our organizational assessment measure is the average assessment given by individual respondents in the focal organization to contact organizations. When the contact organization is the focal point, the sector-based measures refer to the sectoral location and distribution of respondent organizations with ties to them (only three sectors in this case). Network size for contact organizations is the number of respondent organizations that have reported ties to them. Most important, organizational performance is the average assessment of the respondent organization by individual respondents from contact organizations. These last two concepts are useful for evaluation purposes. Evaluations based on selfstudies are generally biased, and even external review committees must work with information provided by the organizations themselves. However, measures based on the aggregated reports of other actors in the system are not subject to this form of systematic bias. Appendix B illustrates the results of this aggregation for the system of international organizations. After the name of the organization, the first column presents the total number of linkages reported with 29 that organization across all three locations of the study. This measure reflects the general level of activity or involvement. CGIAR centers such as ICRAF and CIMMYT have much greater impact on the research systems in these locations than IFDC or IBAH. The second column presents the performance rating for the organization, averaged across all those who reported some contact with that organization. Having ties to a system does not entail that an organization has a positive impact. For instance, CGIAR centers receive high scores in general, but the International Food Policy Research Institute (IFPRI) garners relatively low ratings. Such a measure of assessed performance is a more convincing reflection of the operation of an organization within a designated system of actors than self evaluations of any kind. CONCLUSION Policy formulation on research for development problems must be based on a sound background of knowledge of existing conditions. Since most donors and international organizations are now oriented to the enhancement of research capacity in the developing countries themselves and emphasizes the importance of linkages, we sought to provide a methodology that would begin to address some of the many gaps in our knowledge of the research process. Although there is a large body of work on science and technology for development, very little has been based on systematic, comparative information about the researchers on whom science and technology ultimately depend. With the exception of a few studies based on a highly selective samples and reviews of work visible in the international arena (published in European and Western outlets), there is little systematic evidence regarding the 30 institutional conditions and linkages between institutions. As Farrington and Bebbington emphasize, methods for monitoring and evaluation of research capacity are under-resourced for most national agricultural research systems (NARS), not to mention non-governmental actors (1993). They argue that evidence regarding costs and benefits will remain fragmentary because (1) most NGO efforts are exploratory in nature and intended to produce largely qualitative benefits--such as strengthened capacity for experimentation among farmers; (2) the goals of NARS and NGOs do not coincide. The methodology described here represents a means of generating information on a multi-sectoral national research system that is "objective" to the extent that this is possible, providing relative performance estimates of organizations in different sectors. Whether certain kinds of linkages are associated with improved performance ratings is an empirical question that cannot be addressed here. However, the need for a foundation of information that can be used in the formulation of policy recommendations seems beyond doubt, since the vast literature on capacity building contains little that is both comparative and systematic. An understanding of the contemporary research process in the developing world requires not only that differences between countries, but also the diversity of sectors and types of linkages be taken into account. 31 References{tc \l1 "References} Biervliet, W.E. (1994). Strengthening the capacity of developing countries for research and policy analysis: reflections on a survey of donor policies. In Development and strengthening of research capacity in developing countries, ed. M. Veldhuis, 23-34. The Hague: RAWOO. Farrington, John and Anthony Bebbington. (1993). Reluctant Partners: Non-Governmental Organizations, the State, and Sustainable Agricultural Development. London: Routledge. Schott, Thomas. (1993). World science: globalization of institutions and participation. Science, Technology, and Human Values, 18: 196-208. Shrum, Wesley. (1985). Organized Technology: Networks and Innovation in Technical Systems. West Lafayette, Indiana: Purdue University Press. Shrum, Wesley and Carl Bankston. (1993/94). Organizational and geopolitical approaches to international science and technology networks. Knowledge and Policy, 6: 119-133. Taylor, T. A. (1990). Organization and structure of national agricultural research systems in Anglophone sub-Saharan Africa. ISNAR Working Paper, No. 38, 1-48. Veldhuis, M. (1994). Development and strengthening of research capacity in developing countries. The Hague: RAWOO.